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one-half a standard deviation higher for CATAMA participants than for 
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The Center 



Every child has the capacity to succeed in school and in life. Yet far too many 
children, especially those from poor and minority families, are placed at risk by school 
practices that are based on a sorting paradigm in which some students receive high-expecta- 
tions instruction while the rest are relegated to lower quality education and lower quality 
futures. The sorting perspective must be replaced by a “talent development” model that 
asserts that all children are capable of succeeding in a rich and demanding curriculum with 
appropriate assistance and support. 

The mission of the Center for Research on the Education of Students Placed At Risk 
(CRESPAR) is to conduct the research, development, evaluation, and dissemination needed 
to transform schooling for students placed at risk. The work of the Center is guided by three 
central themes — ensuring the success of all students at key development points, building 
on students’ personal and cultural assets, and scaling up effective programs — and conducted 
through seven research and development programs and a program of institutional activities. 

CRESPAR is organized as a partnership of Johns Hopkins University and Howard 
University, in collaboration with researchers at the University of California at Santa Barbara, 
University of California at Los Angeles, University of Chicago, Manpower Demonstration 
Research Corporation, University of Memphis, Haskell Indian Nations University, and 
University of Houston-Clear Lake. 

CRESPAR is supported by the National Institute on the Education of At-Risk 
Students (At-Risk Institute), one of five institutes created by the Educational Research, 
Development, Dissemination and Improvement Act of 1994 and located within the Office 
of Educational Research and Improvement (OERI) at the U.S. Department of Education. The 
At-Risk Institute supports a range of research and development activities designed to 
improve the education of students at risk of educational failure because of limited English 
proficiency, poverty, race, geographic location, or economic disadvantage. 
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Abstract 



In Talent Development Middle Schools, students needing extra help in mathematics 
participate in the Computer- and Team-Assisted Mathematics Acceleration (CATAMA) 
course. CATAMA is an innovative combination of computer-assisted instruction and 
structured cooperative learning that students receive in addition to their regular math course 
for about ten weeks of the school year. This report presents two studies of CATAMA. The 
first compares growth in math achievement for 96 seventh graders, 48 of whom participated 
in CATAMA for ten weeks and 48 of whom were students of similar prior achievement who 
attended a comparison school where CATAMA is not offered. The second study reports data 
from interviews with CATAMA participants and observations of the program in action. 
Growth in mathematics procedures achievement was about one-half a standard deviation 
higher for CATAMA participants than for students in the comparison sample. High levels 
of student engagement and cooperation were observed among participants. Students liked 
being in CATAMA and working with a partner and a computer to strengthen their procedural 
knowledge and skills. The discussion suggests that CATAMA has many advantages — 
compared to other approaches to providing extra help in math — on several dimensions, 
including cost, capacity, and flexibility. 
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Introduction 



The Talent Development Middle School is a whole-school reform model developed 
by researchers, educators, and experienced curriculum writers at Johns Hopkins University 
and Howard University in collaboration with middle school practitioners. The Talent 
Development Model aims to develop the talent of all students by helping middle schools 
with high concentrations of poverty establish standards-driven curriculum and instruction 
in all major subject areas for all students, intensive extra-help opportunities for students in 
mathematics and reading, innovative approaches to school organization and staffing, and 
focused and sustained professional development opportunities for teachers (Mac Iver, 
Balfanz, Plank, & Ruby, 1998; Plank & Mac Iver, 1998; Mac Iver, & Plank, 1997, Mac Iver, 
Plank, & Balfanz, 1997; Madhere & Mac Iver, 1996; Useem, 1998). Talent Development 
Middle School staff from Johns Hopkins assist schools in phasing in these reforms over a 
three to four year period. This report describes how the Talent Development approach to 
providing extra help in mathematics has been applied in the nation’s first Talent 
Development Middle School in Philadelphia, and describes our ongoing efforts to evaluate 
and refine this approach. 

Extra Help: Who Needs It? 

Middle schools which seek to offer all students a standards-driven mathematics 
curriculum that prepares them for college and features “teaching for meaning” (Knapp, 1 995) 
need effective extra-help programs to provide additional support for students whose 
procedural knowledge and skills are weak compared to local or national norms. Although 
almost all public schools offer some type of extra-help program in mathematics, remarkably 
little data exists concerning the effects of different extra-help opportunities on students’ 
outcomes. Evidence suggests that provision of trained adult tutors who work one-to-one or 
one-to-two with students is effective, especially if these tutors adapt the content and pace of 
instruction to the needs of individual students and if tutoring is provided in addition to 
regular classroom instruction and therefore adds to instructional time (Cooledge & Wurster, 
1985; Devin-Sheehan, Feldman, & Allen, 1976; Mac Iver, 1991; Wasik & Slavin, 1990; 
Wilks & Clarke, 1988). However, most schools cannot afford to hire adult tutors to serve 
students who need extra help, especially if they have many such students. In fact, schools in 
which extra-help opportunities are needed most (e.g., in middle schools in which the average 
achievement of students upon entry to the school is considerably below national norms) are 
the very schools that are least likely to offer one-to-one tutoring. 
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Mac Iver (1991) used data from NELS:88 to investigate the effects of different types 
of extra-help programs in the middle grades on students’ standardized test scores in both 
reading and math. He found that these correlational data suggested that approaches in which 
struggling students receive a substantial extra dose of instruction (e.g., an elective 
replacement class, a Saturday class, or summer classes) were much more effective than less 
intensive approaches (small group pull-out programs and before- or after-school coaching 
classes). Mac Iver (1991) urged researchers to test the replicability of these findings with 
intervention studies that first implement and then evaluate intensive approaches. This report 
presents the results of two such studies. 

In middle schools that have very high proportions of students from low-income 
families, almost all students in the school need extra help in mathematics. This extra help is 
especially important in schools that are beginning implementation of the Talent Development 
blueprint for middle school reform. First, many students are significantly behind local and 
national achievement norms in mathematics and need an opportunity to catch up quickly so 
that they will not be overwhelmed by the challenging learning tasks that they will face once 
a standards-driven curriculum pervades the school. Second, some students need extra help 
because they currently receive their regular mathematics course from a teacher who is not 
yet offering excellent curriculum and instruction in mathematics (e.g., teachers or long-term 
substitutes whose knowledge of math content, pedagogy, or classroom management is weak 
and teachers who have low expectations for their students who maintain a slow pace and 
offer mostly dumbed-down content or review content rather than new content). Third, many 
of the stronger students in the school also need extra help in mathematics in order to increase 
their chances of competing successfully in the broader educational marketplace (e.g., gaining 
admission to the best available high school programs so that they can enter an educational 
trajectory that is likely to lead to high SAT scores and admission to an excellent college or 
university). 

What Kind of Extra Help is Provided? 

In Talent Development Middle Schools, students needing extra help in mathematics 
participate in the Computer- and Team- Assisted Mathematics Acceleration (CAT AM A) 
course. CATAMA is an accelerated learning course that students receive in addition to their 
regular math course. CATAMA replaces an elective course for about ten weeks of the school 
year to provide a substantial “extra dose” of intensive instruction. 

As its name implies, the CATAMA course involves an innovative combination of 
computer-assisted instruction and structured cooperative learning. Different software is used 
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for students from different grade levels. Seventh graders, for example, are assigned an 
individualized sequence of lessons from the SkillsBank 3 computer program, which contains 
100 lessons covering a broad spectrum of proficiencies in math computation, math concepts, 
word problems, geometry, algebra, and thinking skills. They are first given a placement test 
to identify which lessons each student needs and to create partnerships between students who 
need the same sequence of lessons. There are 20 students in each section of CATAMA and 
10 computers in the CATAMA lab. Partners share a computer each day and take turns 
reading aloud the on-screen examples, hints, and explanations and solving the 8 to 10 
problems that accompany each lesson. When a partnership completes a subsection of lessons, 
each student takes an individual quiz to assess individual learning of the concepts and skills 
covered in that subsection. Similarly, when a partnership completes all the lessons in a 
section, each student takes an individual exam to assess his or her mastery. 

Each partnership is paired with another partnership to form a four-person cooperative 
learning team. If one partnership in a team gets stuck, it is the other partnership’s job to help 
them get unstuck. Teammates encourage each other and help each other learn. (For example, 
when a student’s computer is unavailable to him because his or her partner is taking an 
individual quiz or exam on the computer, the student may work with the other two members 
of his team.) Teams that are successful in helping every team member learn receive 
certificates or other modest rewards to recognize the team’s success. 

Although students spend a majority of their time working with their partner on the 
computer, the teacher frequently offers whole-class or small group lessons on specific 
concepts or skills to supplement or reinforce the computer-based instruction. 

This report presents two studies of CATAMA. The first study compares the growth 
in math achievement for 96 seventh graders, 48 of whom participated in the CATAMA 
course for ten weeks during the 1996-1997 school year and 48 of whom were students of 
similar prior achievement who attended the Talent Development Middle School’s 
comparison school where CATAMA is not offered. The second study is a qualitative study 
of the CATAMA program in the 1997-1998 school year and involves interviews with 
CATAMA participants and observations of the program in action. 

These studies are followed by a general discussion of the CATAMA program which 
compares the “elective replacement” approach to providing extra help with other approaches 
along dimensions such as cost-effectiveness and capacity. 
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Study 1 

Growth in Math Achievement by Seventh Graders 
Participating or Not Participating 
in the CATAMA Program 



Participants 

Sixty-five seventh graders participated in the CATAMA program at Central East 
Middle School in Philadelphia during the 1996-1997 school year. Prior mathematics 
achievement data from Spring 1996 on the abbreviated battery of the Stanford 9 achievement 
test were available for 58 of these 65 CATAMA participants. To obtain an estimate of the 
effect of CATAMA on students’ mathematics achievement, we drew a comparison sample 
of seventh graders — with matching prior total mathematics scale scores from Spring 1996 
— from Central East Middle School’s demographically matched comparison school. 1 

We selected our comparison sample by printing out a list of prior achievement data 
for all seventh graders from the comparison school who had completed the mathematics 
battery of the abbreviated version of the Stanford 9 Achievement Test in Spring 1996. 
Students were sorted according to their total mathematics score on this test. For each 
CATAMA participant, we then chose a comparison student whose total mathematics score 
was equal to that of the participant (Main Criterion). If there was more than one student in 
the comparison school whose total mathematics score from Spring 1996 equaled that of the 
CATAMA participant, then we used the following additional criteria in selecting a 
comparison student (in order of priority): 

Additional Criterion 1 : Choose a student who matches the CATAMA 

participant exactly on the two subscales (problem- 
solving and procedures) of the total mathematics 
battery. 

Additional Criterion 2: Choose a student who matches the CATAMA 

participant’s gender. 

If more than one student met the Main Criterion and both of the two Additional 
Criteria, we used a random number table to select among these students. If there were no 
students who met the main criterion — no student whose total mathematics score was an 
exact match to that of the participant — we selected the closest available match as long as 
there was a student within at least 6 points of the participant on the total mathematics scale. 
Once a comparison site student was assigned as a match, he or she was removed from the 
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available pool. Matches were found for 48 of the 58 CATAMA participants who had prior 
achievement data. Thus, the main analyses reported here focus on 48 CATAMA participants 
and a matched sample of 48 nonparticipants from the comparison school. 

Our procedure resulted in the selection of a comparison sample whose prior 
achievement was highly similar to that of CATAMA participants (Mean Math Total Scale 
Scores from Spring 1996: ^comparison sample) = 652 and M<catama participants) = 653, t = .204, 
U ~ -84) 

End-of-Year Mathematics Procedures Scale Scores 

We hoped that analyses of students’ mathematics procedures scale scores from 
Spring 1997 would indicate that holding prior achievement constant, CATAMA participants 
demonstrate greater mastery of mathematics procedures than students in the comparison 
sample. In addition, we thought that the effect of CATAMA participation on mathematics 
procedures achievement might be stronger for students with relatively higher levels of prior 
achievement (the students for whom achieving a substantial boost in mathematics 
proficiency might have a very tangible and salient payoff — allowing them to qualify for a 
selective admission high school program.). 



Table 1. Multiple Regression Model to Predict Seventh Graders’ 
End-of-Year Mathematics Procedures Scale Scores on the 
Stanford 9 Achievement Test (N=96 students) 



Predictor 


B 


Se 


p-value 


Standardized 

Coefficients 


Intercept 


674.57 


3.27 


.00 




PRIORMATHACH 


.62 


.14 


.00 


P = .47 


CATAMA_PARTICIPATION 


12.16 


4.62 


.01 


6 = .49 


PRIOR X CATAMA 


.40 


.20 


.05 





Note. PRIOR_MATH_ACH is grand-mean-centered. R 2 = .46. 



Table 1 summarizes the results from a multiple regression model in which students’ 
end-of-year mathematics procedures scale scores (MATH_PROCEDURES) are predicted 
based on 3 variables: students’ prior mathematics total scale scores from the end of the 
previous school year (PRIOR_MATH_ACH), students’ participation in CATAMA 
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(CATAMA_PARTICIPATION, coded 0 for non-participants, and 1 for participants), and 
a product term representing the interaction of these two variables (PRIOR X CATAMA). 

As expected, student’s prior math achievement is a significant predictor of their end- 
of-year mathematics procedures scale scores (b = .62). As the P coefficient associated with 
this effect indicates, a one standard deviation increase in students’ prior math achievement 
is associated with an increase of .47 standard deviations in their mathematics procedures 
scale scores. 

The coefficient for CATAMA PARTICIPATION (12.16) indicates that mathematics 
procedures achievement scale scores were much higher for typical CATAMA participants 
than for students in the comparison sample. This effect of CATAMA PARTICIPATION is 
expressed in standard deviation units in the final column where the effect size coefficient (6) 
indicates that math procedures achievement scale scores were about one-half of a standard 
deviation higher for the average CATAMA participant than for the average student in the 
matched comparison sample. This effect of CATAMA participation on mathematics 
procedures scale scores translates into an 11 -point advantage in students’ end-of-year 
national percentile ranks — the typical CATAMA participant was boosted to the 54th 
national percentile in math procedures achievement, while the typical student from the 
matched comparison sample reached only the 43rd national percentile. 

Finally, there is also a significant interaction between prior math achievement and 
CATAMA participation (b = .40). To explore the nature of this interaction, we examined 
growth in students’ math procedures scale scores for participants and nonparticipants across 
the entire prior achievement spectrum. This inspection revealed that CATAMA participants 
outgained nonparticipants across the entire prior achievement spectrum, except for the four 
participants and four nonparticipants with the lowest starting points (those whose total math 
scale scores from the previous spring were between 585 and 624). 2 Table 2 summarizes the 
mean growth in math procedures scale scores obtained by participants and nonparticipants 
for these 8 cases (with prior total math scale scores below 625) and for the other 88 cases 
(with scores above 625). For the 88 cases with starting points on the math total scale over 
625, CATAMA participants grew by 28.6 points on average (outgaining nonparticipants by 
almost 1 8 points). The four CATAMA participants with starting points on the math total 
scale that were below 625 also showed dramatic growth (growing 37.3 points on average). 
However, these four CATAMA participants were outgained by their four matches from the 
comparison sample by about 24 points. It may be that the four seventh graders from the 
comparison school with the lowest starting points received one-to-one tutoring or some other 
similarly intense form of help. 



6 




12 



Table 2. Growth in Math Procedures Scale Scores (between Spring 96 and Spring 97) 
for CATAMA Participants and Nonparticipants Whose Prior Total Math 
Achievement Scale Scores Were Greater than 625 or Less than 625 



Group 


n 


Mean Growth 


SD 


Prior Total Math Achievement 
Greater than 625 








CATAMA Participants 


44 


28.6 


28.7 


Nonparticipants 


44 


10.9 


23.0 










Prior Total Math Achievement 
Less than 625 








CATAMA Participants 


4 


37.3 


11.6 


Nonparticipants 


4 


61.5 


8.2 



In summary, the results indicate that CATAMA participants from across the entire 
prior achievement spectrum showed dramatic growth in their understanding and application 
of math procedures. Also, for the vast majority of CATAMA participants, this dramatic 
growth was much larger than the growth observed in similar students from the comparison 
sample. 

Number of Seventh Graders Meeting Minimum Criterion 
on Mathematics Procedures Test for Admission into 
Special Admission High School Programs 

One concrete goal of the CATAMA program is to help greater numbers of seventh 
graders to boost their end-of-year mathematics procedures test scores in mathematics on the 
Stanford 9 to at least the 85th local percentile. Many of the special admission high schools 
in Philadelphia — such as George Washington Carver High School of Engineering and 
Science, Central High School, or Philadelphia High School for Girls — require a student to 
score at this percentile or above at the end of the seventh-grade year in order to be considered 
for admission as a ninth grader. Eighteen of 48 CATAMA participants reached the 85th 
citywide percentile on their mathematics procedures test as opposed to only 7 of the 48 
matched students from the comparison sample (% 2 (1) = 6.54, p =.01). 
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End-of-Year Mathematics Problem-Solving Scale Scores 

Despite the strong positive impact of CATAMA participation on students’ procedural 
knowledge and skills in mathematics, we anticipated no impact of CATAMA on students’ 
performance on the problem-solving subtest of the Stanford 9 because this test emphasizes 
nontraditional competencies such as reasoning in spatial contexts, reasoning with 
proportions, reasoning from graphs, identifying and using patterns and functional 
relationships, understanding variables, expressions, equations, and probability, understanding 
geometric objects and relationships and using geometry in solving problems. These 
competencies figure prominently in the NCTM standards as important proficiencies that have 
not received much attention in traditional curriculum and instruction nor in traditional 
software packages such as Skillsbank 3. These competencies all receive a great deal of 
attention in the standards-driven curriculum that Central East Middle School began 
implementing this school year, the year after Study 1 was conducted. One reason for the 
CATAMA program is to reassure teachers that they can begin focusing more on these 
important new proficiencies in the regular classroom, because students who need extra work 
on their procedural knowledge and skills participate in CATAMA in addition to attending 
a regular math class. 

As expected, multiple regression analyses which control for students’ prior total 
mathematics achievement scale scores indicate that CATAMA participants scored no better 
and no worse than nonparticipants on the problem-solving subtest (b = .17, p =.97). A 
follow-up analysis indicated that this finding of “no benefit of CATAMA" on the problem- 
solving subtest was true across the entire achievement spectrum. That is, there was no 
PRIOR ACHIEVEMENT x CATAMA PARTICIPATION interaction (b = -.03, p =.84). 

For the typical student served, the CATAMA program was an effective “booster shot” 
that assisted students in achieving a dramatic gain of about one-half of a standard deviation 
more than comparison students in their knowledge of mathematical procedures. 
Nevertheless, there are many remaining questions concerning what the CATAMA program 
looks like in actual practice, students’ perceptions of CATAMA, and the cost effectiveness 
of CATAMA relative to alternative approaches. Study 2 (conducted during Winter 1998) and 
the general discussion that follows it address many of these questions. 
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Study 2 

Qualitative Study of CATAMA in Practice 



Observations and interviews were used to examine the CATAMA program and 
explore three critical relationships, those between student and teacher, student and student, 
and student and computer. During the 1997-1998 school year the CATAMA program was 
observed on six occasions by the second author of this report. Each observation consisted of 
viewing the operation of the program during multiple sections in a school day. Each section 
served students in different grade levels (fifth, sixth, and seventh grade). These observations 
were followed by structured interviews with fifth and seventh grade students participating 
in the CATAMA program during the winter term of 1998. 

Observing CATAMA 

Repeated observations of the CATAMA program revealed several consistent features: 
a relaxed but purposeful atmosphere, a strong bond between the students and teacher, and 
high levels of student engagement and cooperation. 

Student-teacher interactions. The CATAMA lab is housed in what was once a 
storage room in the factory that was subsequently converted into Central East Middle School. 
It is a long and narrow space without windows and has a recalcitrant heating system. Despite 
these architectural shortcomings, the CATAMA teacher has made it a cheery place decorated 
with instructional posters conveying information on problem-solving strategies, cooperative 
behavior, and mathematical procedures. Each of the long walls of the room is lined with five 
computers, each with two chairs. At the front of the room are the teacher’s desk, an overhead 
projector, and a blackboard. 

During the observed sessions, the students seemed accustomed to the routine and 
productive climate established by the teacher. Typically they entered quickly, selected a 
white board slate (which is used for scratch work and whole class exercises) from a common 
bin, sat down at their computers, entered their password, and got to work. Once all the 
students were settled at their computers, the teacher frequently used an overhead projector 
to pose a warm-up problem that was linked to the lessons the students were currently 
studying in their math class. With the fifth grade students, this warm-up was followed on 
several occasions by a period of whole class direct instruction. The teacher reports that she 
added this innovation to her routine this year in response to the school’s implementation of 
a more challenging standards-based curriculum. This instruction is often used to preview 
material that will be covered in upcoming lessons in the students’ math class. The teacher 
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reports that one benefit of this preview strategy is that students who have been identified by 
their math teacher as low ability students often become the class experts in new procedures 
(because they have already learned how to do them in CATAMA class). This in turn alters 
students’ and teachers’ perceptions of the students’ mathematical abilities. 

After these opening activities, each student works on the computer software program 
(Cornerstone Math in fifth and sixth grade and Skillsbank 3 in seventh grade) with his or her 
partner for about 30 to 40 minutes per period. Since the partners are paired according to their 
results on a pretest and the software enables students to repeat sections until they are satisfied 
with their progress and have passed a quiz, each partnership in the class is often working on 
different topics and problems. While the students do this, the teacher performs a variety of 
instructive and supportive roles. During the six observations, these included working with 
small groups of students at the blackboard, testing individual fifth graders on their 
multiplication facts, helping students at their computers (this ranged from helping to interpret 
the computer program to modeling alternative solution strategies), monitoring student 
progress, and troubleshooting software and computer glitches. At the end of the period, 
students who did not have sufficient time to begin the next section or complete a quiz or test 
were allowed to play computer math games or non-computer-based math games such as “24." 
These non-computer games were also used when computer or software failures limited the 
number of computers available. 

Two positive types of student-teacher interactions were repeatedly and consistently 
observed. First, although there were periods of conversation among partners and other 
students that were not related to mathematics and occasional behavioral outbursts that would 
force the teacher to call the class to order, there was a very high level of time on task across 
all the students. This high level appears to be the result of at least three factors: (a) a positive 
reward structure established by the teacher in the classroom using a variety of point systems 
to identify and reward partnerships and teams that worked diligently, engaged in productive 
behavior, and helped each other; (b) the teacher’s firm but obviously caring personality; and 
(c) the engaging nature of the work performed in the classroom. 

A second positive feature of student-teacher interaction repeatedly observed was the 
flexible use of resources. The teacher, especially with the fifth graders who had been 
identified as in need of extra help, used a variety of methods and materials to fill in the basic 
mathematical knowledge the students needed and to get them used to the procedures and 
expectations of the new standards-based curriculum that was being implemented. Although 
partnership work using math-focused software still formed the core of CATAMA, it was 
enriched with whole class instruction, small group instruction, fact contests, and 
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mathematical games. These tools were used in a semi-structured way that let students, to 
some extent, move between them as needed. This proved particularly effective with the fifth 
graders, many of whom appeared to have high energy levels and trouble attending to a single 
task for more than ten minutes. By providing them with a variety of tasks and learning 
situations, an environment was created which accommodated their restless energy but kept 
them focused on mathematical learning. 

Student-student and student-computer interactions. The dominant form of 
student-student interaction witnessed in the six observations was between partners. Most of 
this conversation was centered on mathematics and appeared to be productive in nature. 
Partners were observed actively helping each other and demonstrating mathematical 
procedures. They were also observed sharing what might be called unconventional problem- 
solving strategies. These were typically strategies for deducing the correct multiple choice 
answer without fully completing the mathematical procedures called for. Relatively few 
occasions of one student dominating the interactions were observed. This may in part be a 
result of pairing students who have similar pretest scores, and in part a result of the 
individual accountability features built into the program. Even though the students work 
through the lessons as a two-person partnership, they take quizzes and tests individually. 
Thus the incentive for the partnership is not simply to complete the lessons as quickly and 
accurately as possible but for both students to pass their individual quizzes and tests. 

The design ofCATAMA calls for students to work first in partnerships and then in 
teams of four (i.e., if two partners cannot understand something, they are supposed to ask the 
two other members of their team before the teacher). However, relatively little work in teams 
of four was observed. Students at times worked in groups greater than two but these often 
seemed to be based more on friendships than assigned teams. Students working alone (due 
to the absence of a partner) were observed as frequently, if not more, than students working 
in teams of four. 

Student Interviews 

During the winter term of 1998, CATAMA served three populations. Two sections 
of fifth graders identified by teachers as the students in their class most in need of extra help 
were taught, along with one section of sixth grade students similarly selected by one teacher 
from her two sections. CATAMA also served two sections of seventh graders whose sixth 
grade standardized test scores (abbreviated version of the Stanford 9) placed them between 
the 50th and 84th percentile locally. These students were selected in an attempt to boost them 
above the 85th local percentile because scoring above the 85th percentile on the Stanford 9 
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administered in April of seventh grade is one of the requirements for admission to selective 
citywide high schools. It was believed — and supported by evidence obtained in Study 1 — 
that giving these students a double-dose of targeted math instruction prior to the test would 
increase the number of students scoring above the 85th percentile. 

Interviews were conducted with 30 fifth graders and 1 5 seventh graders who attended 
CATAMA on one of two days during the third and fourth weeks of the winter term of 1998. 
The interviews were held in the back of the CATAMA classroom while the class was in 
session. The interviews were conducted by a research assistant using a protocol based on 
observations during the fall and winter and discussions with the CATAMA teacher. The 
interview protocol was designed to query students in four areas: (a) their views on 
participating in CATAMA, (b) their relations with their partners, (c) their views on 
CATAMA instructional approaches, and (d) the impact of CATAMA on their mathematical 
performance and confidence. It consisted of nineteen items which were answered on a Likert 
scale of “not at all or no,” “a little,” and “a lot or yes” and three open-ended items. 



Table 3. Student Views on Participating in CATAMA 



Question 


Grade 


No/Not at All 


A Little 


Yes/A Lot 


Did you want to come to 


5th (n = 30) 


13% 


10% 


77% 


the lab? 


7th (n = 15) 


0% 


7% 


93% 


Do you like the lab? 


5th 


0% 


7% 


93% 




7th 


0% 


0% 


100% 


Do you like working with 


5th 


3% 


0% 


97% 


the computer? 


7th 


0% 


7% 


93% 


Do you like working with 


5th 


3% 


7% 


87% 


a partner? 


7th 


0% 


20% 


80% 


Do you like working in a 


5th 


50% 


7% 


3%* 


team of 4? 


7th 


47% 


27% 


27% 


Would you rather work on 


5th 


83% 


0% 


13% 


the computer by yourself? 


7th 


47% 


20% 


27% 



* Forty percent of the fifth graders said they could not answer the question. 
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Student Views on Participating in CATAMA (Table 3). The interview results 
indicate that almost all students liked participating in CATAMA. This is significant because 
participation in CATAMA is not voluntary on the part of students and replaces an elective 
class. Students were selected based on varying criteria and a letter was sent home asking 
parents for permission to provide CATAMA to the student in lieu of an elective (art, music, 
gym, family life, etc.) for ten weeks. (The parents of a small number of students, about 2%, 
did not consent to their children’s participation in CATAMA.) 

A higher percentage of seventh graders than fifth graders stated that they wanted to 
attend (93% versus 77%), which is not surprising because their participation may have a real 
and immediate consequence (admission to a better high school). Still, three-fourths of the 
fifth graders — students who were identified by their teachers as being those “most in need 
of extra help" — reported that they wanted to come to CATAMA. This indicates that one of 
the goals of the program, designing extra help so that it does not stigmatize students, has 
been achieved. 

The interview results also show that almost all the students liked working with 
computers and that large majorities (80%) liked working with a partner. About half the 
students, however, did not like working in four person teams and between 13% and 27% of 
the students said they would prefer to work by themselves. 

Partner Relations (Table 4). The interview results indicate that students worked 
well together in their partnerships. These results also are consistent with the repeated 
observational findings that students discussed mathematics with each other and helped each 
other. Almost all of the students reported giving help to and receiving help from their 
partners. Although 47% of seventh graders stated that on occasion they let their partners 
provide an answer without explanation, only 13% reported this happening a lot. Fifth graders 
appear not to have understood this question. Eighty-three percent of them gave no answer. 

Instructional Methods (Table 5). The interviews support a number of hypotheses 
on why students liked the CATAMA lab and why high levels of engagement were observed. 
Eighty percent or more of the students interviewed agreed that they liked the lab because you 
can redo sections to get higher scores, get your answer checked right away, and get help 
when you need it. The only area of disagreement between the fifth and seventh graders 
regarded pace. Ninety-three percent of the seventh graders reported that one of the things 
they liked about CATAMA was that you could set your own pace. This view was shared by 
only 57% of the fifth graders. 
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Table 4. Partner Relations 



Question 


Grade 


No/Not at All 


A Little 


Yes/A Lot 


Does your partner help 


5th (n = 30) 


7% 


0% 


90% 


you with things you 
don’t understand? 


7th (n = 15) 


7% 


7% 


87% 


Do you help explain 


5th 


0% 


0% 


97% 


problems to your partner 
if he/she does not 
understand? 


7th 


7% 


13% 


80% 


Do you let your partner 


5th 


7% 


0% 


10%* 


give the right answer 
even if you don’t 
understand? 


7th 


40% 


47% 


13% 



* Eighty-three percent of the fifth graders did not answer (i.e., replied "don’t know” or “don’t understand"). 



Table 5. Views on CATAMA Instructional Methods 



Question 


Grade 


No/Not at All 


A Little 


Yes/A Lot 


What do you like 










about the lab? 










Can redo the section 


5th (n = 30) 


3% 


3% 


93% 


to get a higher score 


7th (n = 15) 


0% 


7% 


93% 


Get the answer 


5th 


10% 


3% 


87% 


checked right away 


7th 


0% 


27% 


79% 


Can set your own pace 


5th 


33% 


10% 


57% 




7th 


1 % 


0% 


93% 


Can get help when 


5th 


0% 


7% 


93% 


you need it 


7th 


7% 


7% 


87% 
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Table 6. Effect of CATAMA on Self-Perception of Mathematical Skill and Confidence 



Question 


Grade 


No/Not at AH 


A Little 


Yes/A Lot 


Is the lab work what you 


5th 


42% 


20% 


37% 


are learning in your math 
class? 


7th 


27% 


33% 


40% 


Compared to before do 


5th 


13% 


7% 


80% 


you understand math 
better? 


7th 


13% 


7% 


80% 


Do you think it’s because 


5th 


13% 


17% 


70% 


of the lab? 


7th 


0% 


40% 


53% 


Compared to before do 


5th 


0% 


13% 


87% 


you feel more confident? 


7th 


7% 


13% 


80% 



Self-Perception of Mathematical Skill and Confidence (Table 6). The students 
disagreed on whether or not their work in CATAMA was related to what they were learning 
in their regular math class. About equal numbers said it was and was not. Actually, parts of 
CATAMA are often directly linked to the material currently being covered in the students’ 
math class (the warm-up activities) and other parts are designed to improve students’ 
knowledge of basic facts and mathematical procedures (the computer-assisted instruction). 

Eighty percent of the students felt that they currently understood math better than 
before and 80 to 90 percent of these students believed that CATAMA had played some role 
in improving their understanding. Fifth graders more than seventh graders (70% versus 53%) 
said that CATAMA had played a large role in their increased understanding. Almost all fifth 
and seventh graders participating in CATAMA (87% and 80%) also reported that they are 
feeling more confident in their mathematical abilities 

Summary. Both the observational study and student interviews indicate that 
CATAMA was a productive and beneficial experience for students. High levels of student 
engagement and cooperation were observed and reported by students. Students liked being 
in CATAMA, working with a partner, and using computers. They appreciated the semi- 
structured but focused environment that has been created by the CATAMA teacher, as well 
as the ability to get immediate and constructive feedback from both her and the computer 
software. Students also believed and liked that CATAMA improved their math skills and 
attitudes. In short, the observations and interviews supported the notion that CATAMA is 
an effective means of providing extra help. 





Discussion 



Study l’s results from the 1996-1997 school year confirm that the Talent 
Development Middle School’s elective replacement class in mathematics helped many 
seventh graders achieve dramatic gains in their procedural knowledge and skills. The typical 
CATAMA participant outperformed students from the comparison sample by almost half a 
standard deviation on the mathematics procedures subtest of the Stanford 9. Further analyses 
revealed that the CATAMA program also helped greater numbers of seventh graders boost 
their end-of-year mathematics procedures test scores on the Stanford 9 to at least the 85th 
local percentile, helping them meet one of the standards for admission at most of 
Philadelphia’s selective admission high school programs. 

Study 2’s results from Winter 1998 suggest that the CATAMA program was 
characterized by productive, on-task student-teacher interactions, student-student inter- 
actions, and student-computer interactions. In addition, students liked being in CATAMA 
and viewed it as a helpful booster shot rather than an unwelcome overdose of mathematics. 
They appreciated the flexible, task-focused learning environment of CATAMA that was 
tailored to their partnership’s learning needs and pace of mastery. 

The elective replacement approach to providing extra help, although found in only 
17% of the nation’s middle schools, has many advantages compared to more common extra- 
help programs (Mac Iver, 1991). Large numbers of students can be served each year. 
Attendance is high because the elective period of academic instruction is part of the regular 
school day. It is not a pullout program, so students do not miss regular academic instruction. 
Also, CATAMA does not stigmatize students as some remedial programs do, but rather is 
viewed as a “plum elective.” Finally, CATAMA is proving to be academically effective, 
while little evidence exists for the effectiveness of other programs, such as extended school 
day or contractual for-profit tutoring. 

An elective replacement approach to extra help is also cost effective. The major 
expenses in creating a CATAMA lab are the cost of a teacher, the computers, and the 
software. In many cases, CATAMA labs can be staffed by reassigning existing staff. Because 
many low-performing schools are also Title I schools, the CATAMA position can often be 
staffed by Title I resource teachers. When staff cannot be reassigned, the cost for a teacher 
ranges from approximately 45 to 65 thousand dollars. Sufficient computers also often exist 
within the building; when they do not, ten computers — adequate for the lab — can be bought 
for ten thousand dollars. The necessary software can be purchased for less than five thousand 
dollars. At the maximum, this means that a CATAMA lab can be operated for 65 thousand 
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dollars a year with one-time start-up costs of 15 thousand dollars. Since a single lab can 
serve three hundred students per year, the per student cost of providing ten weeks of extra 
help is 267 dollars per student in the start-up year and 216 dollars in each following year for 
up to 50 hours of extra help. 

This is a favorable cost compared to two increasingly common alternatives for 
providing extra-help: (a) extended school day or extended school year programs and 
(b) contracting out to for-profit tutoring businesses. Unlike many extended day or after- 
school approaches and also unlike many extended-year or summer school approaches, the 
elective replacement approach does not require schools to keep their building open longer 
or pay their teachers for additional hours. (Of course, many extended-day and extended- year 
programs may not be comparable to CATAMA because they cover many subjects and have 
many purposes beyond improving students’ proficiency in mathematics.) Compared to for- 
profit tutoring approaches, CATAMA classes provide similar services (but with a higher 
student-teacher ratio) at about one-third of the cost. 3 

The relatively low cost of the CATAMA lab and the elective replacement approach 
to extra help in general enables the provision of extra help to a wider range of students within 
a school. Virtually all students in low-performing middle schools need extra help in 
mathematics. For example, in this study, we discussed two overlapping populations of 
students: (1) students from various grade levels in the middle school who had been identified 
either by their teachers or standardized test scores as needing extra help in order to succeed 
at the challenging learning tasks contained in a standards-based mathematics curriculum, and 
(2) a subset of the seventh graders served by CATAMA who had scored between the 50th 
and 84th citywide percentile on standardized tests at the end of sixth grade. These seventh 
graders are in need of extra help on two levels. First, if they can improve their test scores and 
place in the 85th percentile locally, they will meet one of the requirements for admission to 
the city’s best high schools. Second, although these students fall between the 50th and 84th 
percentile locally, many are still performing below national grade level norms. 

A CATAMA lab can serve 300 students per year. This means that in a 5-8 middle 
school of 1,000 students, all of the students can be provided at least 10 weeks of additional 
math instruction during their middle school career and 200 of the students can be served 
twice. CATAMA allows help to be offered to all those who need it, not just to the school’s 
lowest performers. More expensive forms of extra help, like contracting out to outside 
vendors or providing one-to-one tutoring, could be reserved for the lowest-performing 
students (such as the four very low-performing students from the comparison school who 
demonstrated remarkable gains). 
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A final feature of the CATAMA program is its flexibility. It can be used to provide 
both remediation and enrichment. For different groups of students, different mixes of 
computer-based, small group, and whole class instruction can be provided. The software and 
instruction can be tailored to support specific mathematics curriculums and/or to prepare 
students for different types of assessment. The observational and interview data indicate that 
much of the power of the program may be in its mix. The CATAMA lab combines a skilled 
and effective teacher who can provide a wide range of help to diverse students with the use 
of computers and peer collaboration which enable students to work at their own level, 
exercise some control over their learning, receive immediate feedback, and engage in 
mathematical discussions. It is this potent combination that is at the heart of CATAMA’s 
success. 
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Endnotes 



1 After Central East had agreed to become the nation’s first Talent Development Middle 
School, the research office of the School District of Philadelphia identified the other school as 
an appropriate comparison site, because it was very similar to Central East in terms of student 
population and characteristics of the teaching staff. Neither school is a magnet school. Each 
school serves fifth, sixth, seventh, and eighth grades. Each enrolls approximately 1,000 students. 
Over 85 percent of the students from each school come from low income families. Also, many 
of the students have learned English as their second language. At Central East, 53 percent of the 
students spoke a language other than English before they started going to school. At the control 
school, the comparable number is 48 percent. 

Central East’s student body is about 45 percent Hispanic, with most of these students 
being of Puerto Rican descent. Another 24 percent of the students are African-American. About 
13 percent of the students are of Asian descent, primarily Cambodian and Vietnamese. Another 
8 percent are white. The remaining 10 percent identify themselves as belonging to other racial 
or ethnic groups. These other groups include Arab-Americans and biracial students. The control 
school’s student body is similarly diverse although, in comparison with Central East, it has a 
higher proportion of Hispanic and white students, a somewhat lower proportion of African- 
American students, and few Asian- American students. Specifically, 57 percent of the students 
are Hispanic; 16 percent are African-American; 15 percent are white; fewer than 1 percent are 
Asian-American; and the remainder describe themselves as biracial or other. 

2 If the regression model in Table 1 is re-estimated after eliminating these 8 cases with 
the lowest prior achievement, the PRIOR MATH X CATAMA interaction disappears (b = .14, 
p = .61). After this nonsignificant interaction term is removed from the model, the estimated 
effect size (6) associated with CATAMA participation is .60 for these 88 cases. 

3 The CATAMA approach is considerably less expensive than contracting out for tutoring 
services. Sylvan Learning Systems is one of the largest outside providers of extra help in public 
schools. Information provided in a recent news release (Sylvan Learning Systems, Inc., 1998) 
of a contract it has been awarded by the Compton Unified School District in Los Angeles County 
enables a rough comparison between Sylvan’s services and costs and those of CATAMA. The 
press release states “Sylvan transforms classrooms designated for the program by the school 
system into Sylvan Learning Centers, similar to suburban retail locations, by equipping them 
with carpeting, furniture, instructional materials, and computers. Also, Sylvan conducts a series 
of assessment tests to gauge each student’s academic strengths and weaknesses, and tailors 
individual education plans to each student’s needs. Sylvan Learning Centers are staffed to 
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provide for small student/teacher ratios (typically 3 students per instructor), ensuring direct 
personalized instruction for every student.” With the exception of the carpeting and the one-to- 
three staffing level, all of these services are provided by CATAMA. 

According to the published report, the contract with Compton Unified School District has 
a total potential value of 5.4 million dollars over three years. During this time Sylvan will 
provide extra-help in the form of two one-hour sessions per week during and after regular school 
hours throughout the school year and during the summer to 1 ,200 students in seven elementary 
schools and one middle school. This translates into a per student cost of $1,500 for up to 96 
hours of extra help (assuming a 48-week year). This indicates that for equivalent hours of extra- 
help, CATAMA costs about one-third as much as hiring Sylvan. It is possible that outside 
vendors such as Sylvan may be more effective than CATAMA or that a few students may need 
more intensive forms of tutoring. However, given both the high costs of outside tutoring vendors 
compared to CATAMA and CATAMA’s greater capacity, schools and districts should seek out 
hard data on program effectiveness and compare the costs and benefits of an elective replacement 
approach to those of an tutoring vendor before signing a contract with such a vendor. 
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